Quick-look diagnostics
Purpose
The main purpose of this template is to produce a set of quick diagnostic plots you can use during the early stages of model development to get a quick look at how your model fits the data. This template has will only create an html preview of some typical diagnostic plots. It will read in covariates of interest from your spec file but it will not attempt to rename your ETAs or save out any plots.
These plots are not intended to be report quality - please use the diagnostics-report.Rmd template to create diagnostics suitable for your report.
Please remember that these plots are just meant to provide a quick look at your model diagnostics. They are not an exhaustive list of every possible plot and were chosen with this projects aims in mind.
While this should give users examples of plots generated with the most up-to-date packages and methods, we’re always happy to have feedback. If you know of more efficient methods or want to suggest alternative ways of plotting the figures please open an issue with the details.
Set up
Model details - Run number: 102
── Status ─────────────────────────────────
• Finished Running
── Absolute Model Path ───────────────────────────
• /data/projects/bbr-nonmem-poppk-foce/model/pk/102
── YAML & Model Files ───────────────────────────
• /data/projects/bbr-nonmem-poppk-foce/model/pk/102.yaml
• /data/projects/bbr-nonmem-poppk-foce/model/pk/102.ctl
── Description ───────────────────────────────
• Base Model
── Tags ──────────────────────────────────
• two-compartment + absorption
• ETA-CL
• ETA-KA
• ETA-V2
• CLWT-allo
• V2WT-allo
• QWT-allo
• V3WT-allo
• proportional RUV
── Notes ──────────────────────────────────
• 1: Allometric scaling with weight reduces eta-V2 correlation with weight. Will consider additional RUV structures
• 2: Proportional RUV performed best over additive (103) and combined (104)
Dataset: ../../../data/derived/analysis3.csv
Records: 4292 Observations: 3142 Subjects: 160
Objective Function Value (final est. method): 30997.908
Estimation Method(s):
– First Order Conditional Estimation with Interaction
No Heuristic Problems Detected
| parameter_names | estimate | stderr | shrinkage |
|---|---|---|---|
| THETA1 | 0.434 | 0.0629 | |
| THETA2 | 4.12 | 0.0276 | |
| THETA3 | 1.12 | 0.0328 | |
| THETA4 | 4.21 | 0.0192 | |
| THETA5 | 1.29 | 0.0354 | |
| OMEGA(1,1) | 0.221 | 0.0530 | 17.9 |
| OMEGA(2,2) | 0.0827 | 0.00983 | 6.02 |
| OMEGA(3,3) | 0.169 | 0.0197 | 0.587 |
| SIGMA(1,1) | 0.0399 | 0.00123 | 5.28 |
Load Spec and define covariates
The covariates can be defined in several ways, please use whichever you find most convenient:
- You can pass them in through params when you render the Rmd file.
- You can define them using flags in your yspec (expected to be called diagContCov and diagCatCov but you can update it).
- You can define them directly in the Rmd code chunk below.
Please note that the order described above will be preserved so, if for example, you choose to define the covariates with flags in your yspec, you can over ride these choices by passing in a specific set of covariates when rendering the Rmd. Both option 1 and 2 would override the choice of covariates listed in the code below.
Read in data
Read in the model details using read_model. Details stored in the mod object can be used to identify the location of the source data (used in $DATA) - to see how this is done look at the bbr::get_data_path() and bbr::build_path_from_model() helper functions.
After reading in the nonmem dataset and the output dataset they’re joined by a NUM column. This assumes that a row number column (called NUM) was included during data assembly. The idea here is that in NONMEM, you table just NUM and none of the other input data items. They all will get joined back to the nonmem output … even character columns.
The data used in the diagnostic plots has been filtered to only include the observations (i.e. EVID==0). Note that further modifications maybe needed, for example, if BLQ data was included in the model or if the DV was log-transformed. The dataset also converts the categorical covariates of interest to factors using the yspec_add_factors function and details described in the spec file.
The id subset gets the first record per ID. This would usually be the baseline value but consider filtering on a baseline flag if available. Also, if the model includes inter-occassion variaibility (IOV), the occassion variable should be included within the distinct function.
General diagnostic plots
The following plots assume that the preferred x-axis labels are defined here.
DV vs PRED and IPRED
Create plots of DV vs PRED and IPRED for the full dataset
NPDE plots
NPDE vs PRED, time and time after dose.
NPDE vs continuous covariates
## $AGE
##
## $WT
##
## $ALB
##
## $EGFR
NPDE vs categorical covariates.
## $STUDY
##
## $RF
##
## $CP
##
## $DOSE
NPDE density histogram
CWRES vs PRED, time and time after dose
CWRES qq and density plot
EBEs-based diagnostics
ETA pairs plot
Continuous covariate plots
ETA vs continous covariates
Note that here we use a function that maps over the ETAs (not the covariates) because the purpose of these plots was to determine whether there were any trends in the covariates for a given ETA. This may need to be edited to address different study specific questions.
## [[1]]
##
## [[2]]
##
## [[3]]
Continous covariate pairs plot
Categorical covariate plots
ETA vs categorical covariates
Note that here we use a function that maps over the covariates (not the ETAs) because the purpose of these plots was to determine whether there is any difference in the distribution of ETAs across studies, dosing groups and disease states. This should be updated to reflect the questions you’re trying to address.
## $STUDY
##
## $RF
##
## $CP
##
## $DOSE
Session details
It is considered good practice to include these details at the end of all rmd scripts
Sys.getenv("AMI_NAME")## [1] "metworx-21.08-2-202202030432-b1e4ce4"sessioninfo::session_info()## ─ Session info ────────────────────────────────────────────────────────────────────────────────────────────
## setting value
## version R version 4.1.1 (2021-08-10)
## os Ubuntu 18.04.5 LTS
## system x86_64, linux-gnu
## ui RStudio
## language (EN)
## collate en_US.UTF-8
## ctype en_US.UTF-8
## tz America/New_York
## date 2022-03-31
## rstudio 1.4.1717.3 Juliet Rose (server)
## pandoc 2.11.4 @ /usr/lib/rstudio-server/bin/pandoc/ (via rmarkdown)
##
## ─ Packages ────────────────────────────────────────────────────────────────────────────────────────────────
## package * version date (UTC) lib source
## assertthat 0.2.1 2019-03-21 [1] CRAN (R 4.1.1)
## backports 1.4.1 2021-12-13 [1] CRAN (R 4.1.1)
## bbr * 1.3.1 2022-03-24 [1] MPNDEV (R 4.1.1)
## bit 4.0.4 2020-08-04 [1] CRAN (R 4.1.1)
## bit64 4.0.5 2020-08-30 [1] CRAN (R 4.1.1)
## broom 0.7.12 2022-01-28 [1] CRAN (R 4.1.1)
## bslib 0.3.1 2021-10-06 [1] CRAN (R 4.1.1)
## cellranger 1.1.0 2016-07-27 [1] CRAN (R 4.1.1)
## checkmate 2.0.0 2020-02-06 [1] CRAN (R 4.1.1)
## cli 3.2.0 2022-02-14 [1] CRAN (R 4.1.1)
## colorspace 2.0-3 2022-02-21 [1] CRAN (R 4.1.1)
## crayon 1.5.0 2022-02-14 [1] CRAN (R 4.1.1)
## data.table 1.14.2 2021-09-27 [1] CRAN (R 4.1.1)
## DBI 1.1.2 2021-12-20 [1] CRAN (R 4.1.1)
## dbplyr 2.1.1 2021-04-06 [1] CRAN (R 4.1.1)
## diffobj 0.3.5 2021-10-05 [1] CRAN (R 4.1.1)
## digest 0.6.29 2021-12-01 [1] CRAN (R 4.1.1)
## dplyr * 1.0.8 2022-02-08 [1] CRAN (R 4.1.1)
## ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.1.1)
## evaluate 0.15 2022-02-18 [1] CRAN (R 4.1.1)
## fansi 1.0.2 2022-01-14 [1] CRAN (R 4.1.1)
## farver 2.1.0 2021-02-28 [1] CRAN (R 4.1.1)
## fastmap 1.1.0 2021-01-25 [1] CRAN (R 4.1.1)
## forcats * 0.5.1 2021-01-27 [1] CRAN (R 4.1.1)
## fs 1.5.2 2021-12-08 [1] CRAN (R 4.1.1)
## generics 0.1.2 2022-01-31 [1] CRAN (R 4.1.1)
## GGally 2.1.2 2021-06-21 [1] CRAN (R 4.1.1)
## ggplot2 * 3.3.5 2021-06-25 [1] CRAN (R 4.1.1)
## glue * 1.6.2 2022-02-24 [1] CRAN (R 4.1.1)
## gridExtra 2.3 2017-09-09 [1] CRAN (R 4.1.1)
## gridGraphics 0.5-1 2020-12-13 [1] CRAN (R 4.1.1)
## gtable 0.3.0 2019-03-25 [1] CRAN (R 4.1.1)
## haven 2.4.3 2021-08-04 [1] CRAN (R 4.1.1)
## here * 1.0.1 2020-12-13 [1] CRAN (R 4.1.1)
## highr 0.9 2021-04-16 [1] CRAN (R 4.1.1)
## hms 1.1.1 2021-09-26 [1] CRAN (R 4.1.1)
## htmltools 0.5.2 2021-08-25 [1] CRAN (R 4.1.1)
## httr 1.4.2 2020-07-20 [1] CRAN (R 4.1.1)
## jquerylib 0.1.4 2021-04-26 [1] CRAN (R 4.1.1)
## jsonlite 1.8.0 2022-02-22 [1] CRAN (R 4.1.1)
## knitr * 1.37 2021-12-16 [1] CRAN (R 4.1.1)
## labeling 0.4.2 2020-10-20 [1] CRAN (R 4.1.1)
## lattice 0.20-45 2021-09-22 [1] CRAN (R 4.1.1)
## lifecycle 1.0.1 2021-09-24 [1] CRAN (R 4.1.1)
## logitnorm 0.8.38 2021-01-07 [1] CRAN (R 4.1.1)
## lubridate 1.8.0 2021-10-07 [1] CRAN (R 4.1.1)
## magrittr 2.0.2 2022-01-26 [1] CRAN (R 4.1.1)
## Matrix 1.4-0 2021-12-08 [1] CRAN (R 4.1.1)
## mgcv 1.8-39 2022-02-24 [1] CRAN (R 4.1.1)
## modelr 0.1.8 2020-05-19 [1] CRAN (R 4.1.1)
## mrggsave * 0.4.3 2022-01-13 [1] MPNDEV (R 4.1.1)
## munsell 0.5.0 2018-06-12 [1] CRAN (R 4.1.1)
## nlme 3.1-155 2022-01-13 [1] CRAN (R 4.1.1)
## patchwork * 1.1.1 2020-12-17 [1] CRAN (R 4.1.1)
## pillar 1.7.0 2022-02-01 [1] CRAN (R 4.1.1)
## pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.1.1)
## PKPDmisc * 3.0.0 2022-01-13 [1] MPNDEV (R 4.1.1)
## plyr 1.8.6 2020-03-03 [1] CRAN (R 4.1.1)
## pmplots * 0.3.4 2022-03-24 [1] MPNDEV (R 4.1.1)
## pmtables * 0.4.1 2022-03-24 [1] MPNDEV (R 4.1.1)
## prettyunits 1.1.1 2020-01-24 [1] CRAN (R 4.1.1)
## processx 3.5.2 2021-04-30 [1] CRAN (R 4.1.1)
## progress 1.2.2 2019-05-16 [1] CRAN (R 4.1.1)
## ps 1.6.0 2021-02-28 [1] CRAN (R 4.1.1)
## purrr * 0.3.4 2020-04-17 [1] CRAN (R 4.1.1)
## R6 2.5.1 2021-08-19 [1] CRAN (R 4.1.1)
## RColorBrewer 1.1-2 2014-12-07 [1] CRAN (R 4.1.1)
## Rcpp 1.0.8.3 2022-03-17 [1] CRAN (R 4.1.1)
## readr * 2.1.2 2022-01-30 [1] CRAN (R 4.1.1)
## readxl 1.3.1 2019-03-13 [1] CRAN (R 4.1.1)
## renv 0.13.2 2021-03-30 [1] CRAN (R 4.1.1)
## reprex 2.0.1 2021-08-05 [1] CRAN (R 4.1.1)
## reshape 0.8.8 2018-10-23 [1] CRAN (R 4.1.1)
## rlang 1.0.2 2022-03-04 [1] CRAN (R 4.1.1)
## rmarkdown * 2.13 2022-03-10 [1] CRAN (R 4.1.1)
## rprojroot 2.0.2 2020-11-15 [1] CRAN (R 4.1.1)
## rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.1.1)
## rvest 1.0.2 2021-10-16 [1] CRAN (R 4.1.1)
## sass 0.4.0 2021-05-12 [1] CRAN (R 4.1.1)
## scales 1.1.1 2020-05-11 [1] CRAN (R 4.1.1)
## sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.1.1)
## stringi 1.7.6 2021-11-29 [1] CRAN (R 4.1.1)
## stringr * 1.4.0 2019-02-10 [1] CRAN (R 4.1.1)
## tibble * 3.1.6 2021-11-07 [1] CRAN (R 4.1.1)
## tidyr * 1.2.0 2022-02-01 [1] CRAN (R 4.1.1)
## tidyselect 1.1.2 2022-02-21 [1] CRAN (R 4.1.1)
## tidyverse * 1.3.1 2021-04-15 [1] CRAN (R 4.1.1)
## tzdb 0.2.0 2021-10-27 [1] CRAN (R 4.1.1)
## utf8 1.2.2 2021-07-24 [1] CRAN (R 4.1.1)
## vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.1.1)
## vroom 1.5.7 2021-11-30 [1] CRAN (R 4.1.1)
## withr 2.5.0 2022-03-03 [1] CRAN (R 4.1.1)
## xfun 0.30 2022-03-02 [1] CRAN (R 4.1.1)
## xml2 1.3.3 2021-11-30 [1] CRAN (R 4.1.1)
## xtable 1.8-4 2019-04-21 [1] CRAN (R 4.1.1)
## yaml * 2.3.5 2022-02-21 [1] CRAN (R 4.1.1)
## yspec * 0.5.1.9100 2022-03-16 [1] MPNDEV (R 4.1.1)
##
## [1] /data/projects/bbr-nonmem-poppk-foce/renv/library/R-4.1/x86_64-pc-linux-gnu
## [2] /tmp/RtmpeGfGAr/renv-system-library
##
## ───────────────────────────────────────────────────────────────────────────────────────────────────────────bbr::bbi_version()## [1] "3.1.1"